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PREFACE 


The  bibliography  presented  on  the  following  pages  is  a  compilation 
of  recent  publications  by  some  of  the  Rand  researchers  in  the  Informa¬ 
tion  Processing  Systems  research  program.  The  bibliography  is  selec¬ 
tive,  comprising  only  those  papers  concerned  with  the  broad  range  of 
topics  in  Cognitive  Science.  These  topics  include  the  modeling  of  com¬ 
plex  human  behaviors,  the  construction  of  intelligent  computer  systems, 
and  the  representation  of  knowledge  in  human  and  computer  memories.  The 
research  draws  upon  the  disciplines  of  cognitive  psychology,  educational 
psychology,  computer  science,  artificial  intelligence,  and  linguistics. 

The  bibliography  is  organized  into  two  sections.  The  first  sectiC^ 
contains  abstracts  for  papers  published  between  April  1979  and  December 
1980.  The  second  section  contains  a  list  of  titles  published  prior  to 
April  1979.  Rand  Papers  P-5950  and  P-5950/1  contain  abstracts  for  these 
earlier  papers. 


The  titles  are  organized  alphabetically,  with  an  author  index  fol¬ 
lowing  the  list  of  publications.  The  Rand  researchers  represented  in 
this  bibliography  are  Sarah  Goldin,  Barbara  Hayes-Roth,  Frederick 
Hayes-Roth,  Philip  Klahr,  Dave  McArthur,  Cathleen  Stasz,  Perry 
Thorndyke,  Don  Waterman,  Clair ice  Veit,  Keith  Wescourt,  and  Robert  Wes¬ 
son.  Requests  for  reprints  of  Rand  publications  may  be  addressed  to  the 
Rand  Publications  Department,  1700  Main  Street,  Santa  Monica,  California 
90406.  Requests  for  reprints  of  journal  articles  or  book  chapters 
should  be  addressed  directly  to  the  author. 
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RESEARCH  IN  COGNITIVE  SCIENCE  BY  RAND  STAFF:  1979-1980 


109.  Ertnan,  L.  D. ,  Hayes-Roth,  F.,  Lesser,  V.  R. ,  and  Reddy,  D.  R.  The 
Hearsay-II  speech-understanding  system:  Integrating  knowledge 
to  resolve  uncertainty.  Computing  Surveys ,  Vol .  12,  No.  2,  June 
1980. 

The  Hearsay-II  system,  developed  during  the  DARPA-sponsored 
five-year  speech-understanding  research  program,  represents  both 
a  specific  solution  to  the  speech-understanding  problem  and  a 
general  framework  for  coordinating  independent  processes  to 
achieve  cooperative  problem-solving  behavior.  Discussed  in  this 
paper  are  the  characteristics  of  the  speech  problem  in  particu¬ 
lar,  the  special  kinds  of  problem-solving  uncertainty  in  that 
domain,  the  structure  of  the  Hearsay-II  system  developed  to  cope 
with  that  uncertainty,  and  the  relationship  between  Hearsay-II 's 
structure  and  those  of  other  speech-understanding  systems.  The 
paper  is  intended  for  the  general  computer  science  audience  and 
presupposes  no  speech  or  artificial  intelligence  background. 


110.  Faught,  W.  S.,  Klahr,  P.,  and  Martins,  G.  R.  An  artificial  intel¬ 
ligence  approach  to  large-scale  simulation.  Proceedings  of  the 
1980  Summer  Simulation  Conference,  Seattle,  1980,  231-236. 

The  authors  present  a  new  approach  to  the  design  of  large-scale 
simulators.  Techniques  developed  in  the  area  of  Artificial 
Intelligence,  particularly  in  the  development  of  "expert  sys¬ 
tems,"  are  relevant  to  the  design  of  simulators  that  model 
decision-making  behavior.  They  have  adapted  and  extended  these 
techniques  in  ROSS,  a  Rule-Oriented  Simulation  System  that  simu¬ 
lates  air  penetration  battles. 


111.  Faught,  W.  S.,  Waterman,  D.  A.,  Rosenschein,  S.  J.,  Gorlin,  D.  M., 
and  Tepper,  S.  J.  EP-2,  A  prototype  exemplary  programming  sys¬ 
tem.  Proceedings  of  the  ACM  Annual  Conference,  Detroit,  1979, 
135-142. 

This  report  describes  the  design  and  implementation  of  the  Exem¬ 
plary  Programming  (EP)  system  that  allows  software  to  be  created 
by  example.  The  EP  paradigm  is  as  follows:  The  user  performs 
some  interactive  task  on  a  computer.  The  EP  system  watches  over 
the  user's  shoulder,  recording  the  interaction  between  the  user 
and  the  system  he  is  using.  When  the  task  is  done,  EP  con¬ 
structs  an  algorithm  or  high-level  model  of  the  interaction. 

Part  of  this  construction  may  involve  questions  to  the  user  or 
advice  from  the  user.  EP  constructs  a  program  (agent)  from  the 
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model  and  stores  it  in  a  library  for  subsequent  use.  A  critique 
and  suggestions  for  the  next  version  of  the  EP  system  are 
included. 


112.  Goldin,  S.  E.,  and  Thorndyke,  P.  W.  (Eds.).  Improving  team  perfor¬ 
mance;  Proceedings  of  the  Rand  Team  Performance  Workshop.  Rand 
Report  R-2606-ONR,  August  1980. 

This  paper  reports  the  proceedings  of  the  Rand  Workshop  on  Team 
Performance  sponsored  by  the  Office  of  Naval  Research.  This 
workshop  considered  options  for  future  research  on  teams  from 
the  perspectives  of  a  number  of  different  disciplines.  Experts 
in  the  domains  of  gaming  and  simulation,  organizational  theory, 
small  group  processes,  cognitive  psychology,  training  and 
instruction,  decision  theory,  heuristic  modeling  and  human 
engineering  presented  views  on  potential  contributions  of  their 
disciplines  to  team  research.  Discussions  among  workshop  parti¬ 
cipants  focused  on  a  range  of  issues,  including  current  problems 
in  the  performance  of  Navy  teams,  the  desirability  of  studying 
teams  in  their  operational  environments,  the  need  for  specifying 
training  objectives  prior  to  instructional  design,  and  the  need 
for  designing  team  research  programs  prior  to  establishing  a 
dedicated  team  research  facility.  A  number  of  broad  areas  seems 
to  offer  promise  for  future  research:  team  performance  require¬ 
ments,  team  structure,  team  communications,  training  techniques, 
and  organizational  determinants  of  team  performance. 


113.  Hayes-Roth,  B.  Estimation  of  time  requirements  during  planning: 
Interactions  between  motivation  and  cognition.  Rand  Note  N- 
1581-ONR,  November  1980. 

Human  planners  show  a  strong  tendency  to  underestimate  the  time 
required  for  planned  tasks.  In  addition,  time  stress  increases 
this  tendency.  The  more  difficult  it  is  to  accomplish  all  tasks 
under  consideration  in  the  available  time,  the  stronger  the  ten¬ 
dency  to  underestimate  time  requirements.  In  addition  to  docu¬ 
menting  these  effects,  this  Note  provides  evidence  for  two 
underlying  factors.  A  cognitive  factor  refers  to  people's  ten¬ 
dency  to  plan  at  high  levels  of  abstraction.  Because  they  fail 
to  enumerate  all  time-consuming  components  of  planned  tasks, 
they  systematically  underestimate  the  time  required  to  perform 
the  tasks  as  wholes.  A  motivational  factor  refers  to  people's 
desire  to  accomplish  all  or  most  of  the  tasks  under  considera¬ 
tion.  This  motivation  biases  them  to  underestimate  required 
times.  The  Note  concludes  with  a  discussion  of  methods  for 
correcting  underestimation  of  time  requirements  during  planning. 


114.  Hayes-Roth,  B.  Flexibility  in  executive  strategies.  Rand  Note  N- 
1170-0NR,  September  1980. 

Executive  strategies  determine  the  allocation  of  cognitive 
resources  during  problem-solving.  Earlier  research  has  sug¬ 
gested  that  people  can  adopt  alternative  strategies  for  solving 
particular  problems.  This  paper  examines  an  "opportunistic" 
model  of  executive  strategies  and  evaluates  some  of  its  predic¬ 
tions  for  performance  of  an  errand-planning  task.  Five  experi¬ 
ments  confirmed  that  (a)  people  can  adopt  different  strategies 
for  this  task;  (b)  people  can  learn  new  strategies  from  explicit 
instruction  or  from  experience;  (c)  problem  characteristics  can 
influence  which  strategy  people  adopt;  and  (d)  adopted  strategy 
interacts  with  problem  characteristics  to  determine  planning 
time  and  number  and  importance  of  planned  errands.  The  results 
also  suggested  that  some  people  have  a  proclivity  to  adopt  a 
particular  strategy  and  resist  adopting  a  new  one.  Implications 
of  the  results  and  desirable  properties  of  the  model  are  dis¬ 
cussed. 


115.  Hayes-Roth,  B.  Projecting  the  future  for  situation  assessment  and 
planning:  A  cognitive  analysis.  Rand  Note  N-1600-AF,  November 
1980 

Situation  assessment  and  planning  rely  heavily  on  the 
decisionmaker's  ability  to  project  future  conditions  in  the 
environment  and  the  impact  of  tentative  planned  actions  on  those 
conditions.  This  Note  presents  a  cognitive  analysis  of  the 
future  projection  process.  It  identifies  three  projection 
strategies--retrieval  from  experience,  formal  analysis,  and  men¬ 
tal  slmulation--and  characterizes  their  strengths  and 
weaknesses.  It  also  discusses  the  impact  of  three  general  cog¬ 
nitive  factors — operation  at  different  levels  of  abstraction, 
motivational  factors,  and  attribution  problems--on  the  future 
projection  process.  Based  on  this  analysis,  the  Note  proposes  a 
cognitive  technology  for  developing  and  training  effective 
future  projection  processes. 


116.  Hayes-Roth,  B.,  with  S.  Cammarata,  S.  E.  Goldin,  F.  Hayes-Roth,  S. 
Rosenschein,  and  P.  W.  Thorndyke.  Human  planning  processes. 

Rand  Report  R-2670-0NR,  November  1980. 

This  Report  summarizes  a  three-year  research  project  investigat¬ 
ing  the  cognitive  processes  underlying  human  planning  behavior. 
The  project  focused  on  problems  analogous  to  the  naval  tactical 
planning  problem:  How  should  the  decisionmaker  move  force  units 
from  their  current  locations  to  particular  task  force  objec¬ 
tives?  Major  project  tasks  included:  developing  a  cognitive 
model  of  the  planning  process,  implementing  the  model  as  a 
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computer  simulation  and  evaluating  its  performance,  conducting 
empirical  research  to  test  the  basic  assumptions  of  the  model 
and  to  identify  important  task  and  individual  difference  factors 
that  impact  on  the  planning  process.  Based  on  the  results  of 
these  research  efforts,  the  project  produced  a  preliminary  set 
of  guidelines  for  improving  the  selection  and  training  of 
planners  and  for  designing  effective  planning  aids. 

117.  Hayes-Roth,  B.,  and  Goldin,  S.  Individual  differences  in  planning 
processes.  Rand  Note  N-1488-0NR,  June  1980. 

This  Note  reports  an  initial  investigation  of  individual  differ¬ 
ences  in  planning.  The  research  reported  here  focuses  on  the 
analysis  of  thinking-aloud  protocols  produced  by  five  subjects 
as  they  performed  a  set  of  errand-planning  tasks .  The  protocols 
were  coded  into  content  categories  suggested  by  the  opportunis¬ 
tic  planning  model  developed  by  Hayes-Roth  and  Hayes-Roth 
(1979).  Final  plans  were  evaluated  against  a  set  of  normative 
criteria,  such  as  route  efficiency  and  temporal  realism.  Then 
the  occurrence  of  various  content  categories  in  the  protocols 
was  related  to  individual  subjects'  planning  scores  in  an  effort 
to  describe  patterns  of  category  usage  that  correlated  with 
planning  skill. 

This  analysis  highlighted  a  number  of  differences  between  good 
and  poor  planners.  Good  planners  used  the  available  set  of 
decision  categories  more  extensively  than  poor  planners.  In 
particular,  good  planners  made  more  decisions  establishing  cri¬ 
teria  for  plan  generation  and  evaluation,  more  decisions  con¬ 
trolling  allocation  of  cognitive  resources  during  planning,  and 
more  decisions  assessing  data  relevant  to  planned  actions.  Good 
planners  also  made  more  decisions  at  higher  levels  of  abstrac¬ 
tion,  especially  decisions  concerning  the  intended  outcomes  of 
the  plan  and  decisions  concerning  an  overall  temporal-spatial 
design  for  the  plan. 

Good  planners  also  differed  from  poor  planners  in  the  detailed 
content  of  their  decisions.  They  more  frequently  reviewed  and 
evaluated  previous  decisions.  They  were  more  likely  to  compare 
alternative  plan  actions.  They  were  more  sensitive  to  con¬ 
straints  on  when  certain  errands  could  be  done  and  to  the 
existence  of  spatial  clusters  of  errands.  They  had  larger 
repertoires  of  decision  types  governing  general  plan  charac¬ 
teristics.  They  generated  a  greater  number  and  variety  of  deci¬ 
sions  determining  the  focus  of  attention.  Finally,  although 
good  planners  and  poor  planners  used  essentially  the  same  cri¬ 
teria  for  generating  and  evaluating  their  plans,  good  planners 
used  most  of  these  criteria  more  frequently  than  did  poor 
planners . 
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118.  Hayes-Roth,  B.,  and  Hayes-Roth,  F.  A  cognitive  model  of  planning. 
Cognitive  Science.  Vol.  3,  1979,  275-310. 

This  paper  presents  a  cognitive  model  of  the  planning  process. 
The  model  generalizes  the  theoretical  architecture  of  the 
Hearsay-II  system.  Thus,  it  assumes  that  planning  comprises  the 
activities  of  a  variety  of  cognitive  "specialists."  Each  spe¬ 
cialist  can  suggest  certain  kinds  of  decisions  for  incorporation 
into  the  plan  in  progress.  These  include  decisions  about:  (a) 
how  to  approach  the  planning  problem;  (b)  what  knowledge  bears 
on  the  problem;  (c)  what  kinds  of  actions  to  try  to  plan;  (d) 
what  specific  actions  to  plan;  and  (e)  how  to  allocate  cognitive 
resources  during  planning.  Within  each  of  these  categories, 
different  specialists  suggest  decisions  at  different  levels  of 
abstraction.  The  activities  of  the  various  specialists  are  not 
coordinated  in  any  systematic  way.  Instead,  the  specialists 
operate  opportunistically,  suggesting  decisions  whenever  promis¬ 
ing  opportunities  arise.  The  paper  presents  a  detailed  account 
of  the  model  and  illustrates  its  assumptions  with  a  "thinking 
aloud"  protocol.  It  also  describes  the  performance  of  a  com¬ 
puter  simulation  of  the  model.  The  paper  contrasts  the  proposed 
model  with  successive  refinement  models  and  attempts  to  resolve 
apparent  differences  between  the  two  points  of  view. 


119.  Hayes-Roth,  B.,  and  Thorndyke.  P.  W.  Decisionmaking  during  the 
planning  process.  Rand  Note  N-1213-0NR,  November  1980. 

Planning  requires  an  individual  to  make  a  series  of  decisions 
about  an  intended  course  of  action.  This  paper  evaluates  two 
major  assumptions  of  an  "opportunistic"  model  of  the  planning 
process:  (a)  that  planners  make  decisions  at  different  levels 

of  abstraction;  (b)  that  prior  decisions  constrain  the  choice  of 
subsequent  decisions;  and  (c)  that  planners  generate  decision 
sequences  opportunistically.  The  results  of  three  experiments 
supported  these  assumptions.  In  Experiment  1,  subjects'  sort¬ 
ings  of  statements  representing  different  planning  decisions 
confirmed  the  postulated  levels  of  abstraction.  In  Experiment 
2,  particular  prior  decisions  influenced  subjects'  choice 
between  alternative  subsequent  decisions.  In  Experiment  3,  dif¬ 
ferent  prior  decisions  influenced  subjects  to  choose  subsequent 
decisions  at  either  higher  or  lower  levels  of  abstraction. 


120.  Hayes-Roth,  F.  Distinguishing  theories  of  representation:  A  cri¬ 
tique  of  Anderson's  "Arguments  concerning  mental  imagery." 
Psychological  Review,  1979,  Vol.  86,  No.  A,  376-382.  Also 
issued  as  Rand  Paper  P-6180,  1978. 


In  a  recent  article,  Anderson  argued  that  behavioral  data  could 
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not  distinguish  alternative  theories  of  memory  representations. 
The  theorem  Anderson  proved  does  not  support  his  conclusions. 
Five  fallacies  in  his  argument  are  criticized;  (a)  Behavioral 
data  cannot  distinguish  alternative  models;  (b)  an  invertible 
function  relates  corresponding  states  of  alternative  representa¬ 
tion  theories;  (c)  propositional  models  can  account  for  data 
supporting  image  models  and  thus,  warrant  equal  credibility;  (d) 
propositional  descriptions  constitute  parsimonious  models;  and 
(e)  unless  explicitly  operationalized,  image  models  cannot  sup¬ 
port  theoretical  or  empircal  research.  Contrary  to  these 
beliefs,  representational  differences  can  qualitatively  affect 
performance,  and  such  important  differences  apparently  charac¬ 
terize  the  analog  class  of  imagery  models. 


121.  Hayes-Roth,  F.  Matching  and  abstraction  in  knowledge  systems. 

Rand  Paper  P-6440,  January  1980. 

A  briefing  presented  at  the  Symposium  on  Artificial  Intelligence 
in  Information  Science  during  the  1979  Annual  Meeting  of  the 
American  Society  for  Information  Science  (ASIS)  in  Minneapolis. 
Discusses  the  creation  of  a  knowledge  system  and  problems  relat¬ 
ing  to  it.  Twenty  examples  are  illustrated  and  discussed  pro¬ 
viding  details  on  knowledge  systems  and  data  representation. 
Topics  include;  matching  and  abstraction  in  knowledge  systems, 
knowledge  bases,  information  retrieval,  partial  matches,  and 
future  research  issues. 


122.  Hayes-Roth,  F.  Understanding  mental  syntaxy:  Interpretive  meta¬ 
phors  vs.  explanatory  models.  Open  Peer  Commentary  of  On  the 
demystification  of  mental  imagery  by  S.  Kosslyn,  S.  Pinker,  G. 
Smith  and  S.  Shwartz,  The  Behavioral  and  Brain  Sciences ,  1979, 
Vol.  2,  53-554. 


123.  Hayes-Roth,  F. ,  Klahr,  P.,  and  Mostow,  D.  J.  Advice-taking  and 
knowledge  refinement:  An  iterative  view  of  skill  acquisition. 
Rand  Paper  P-6517,  July  19B0.  (To  appear  in  J.  R.  Anderson 
(ed.).  Skill  acquisition  and  development ,  Lawrence  Erlbaum  Asso¬ 
ciates,  Hillsdale,  New  Jersey,  1981.) 

This  paper  considers  knowledge  acquisition  and  skill  development 
as  an  iterative  process.  In  the  first  phase,  an  initial  capa¬ 
bility  is  achieved  by  understanding  instructions  and  following 
advice.  Once  a  person  (or  machine)  tries  out  its  new  knowledge, 
a  variety  of  potential  problems  and  learning  opportunities 
arise.  These  stimulate  refinements  to  previous  knowledge  which, 
in  turn,  reinitiate  the  entire  cycle.  This  learning  paradigm 
has  led  to  new  work  in  knowledge  representation,  operationaliza¬ 
tion,  expectation-driven  bug  detection,  and  knowledge  refinement 
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techniques.  This  paper  explains  how  advice  is  converted  into 

operational  behavior,  how  unexpected  or  undesirable  behavioral  ] 

outcomes  stimulate  learning  efforts,  and  how  the  bugs  responsi¬ 
ble  can  be  diagnosed  and  repaired.  These  phenomena  are  illus¬ 
trated  with  examples  of  human  and  machine  skill  development  in  a  ' 

familiar  card  game.  The  proposed  learning  methods  provide  a 
basis  for  a  deeper  understanding  of  skilled  behavior  and  its 

development  than  previously  possible.  j’ 

124.  Hayes-Roth,  F. ,  Klahr,  P. ,  and  Mostow,  D.  J.  Knowledge  acquisi¬ 
tion,  knowledge  programming,  and  knowledge  refinement.  Rand  ' 

Report  R-2S40-NSF,  November  1979.  | 


Principal  findings  and  recommendations  of  a  two-year  study  of 
machine-aided  knowledge  acquisition.  The  report  discusses  the 
transfer  of  expertise  from  humans  to  machines,  as  well  as  the 
functions  of  planning,  debugging,  knowledge  refinement,  and 
autonomous  machine  learning.  The  research  method  emphasizes 
iterative  refinement  of  knowledge  in  response  to  actual  experi¬ 
ence.  A  machine's  "knowledge"  is  acquired  from  a  human,  who 
provides  concepts,  constraints,  and  problem-solving  heuristics 
to  define  some  minimal  level  of  performance.  Semi-automatic 
methods  convert  the  initial  knowledge  into  a  working  program 
whose  resulting  behaviors  can  be  used  to  diagnose  problems  and 
design  refinements.  Methods  formulated  here  may  reduce  or  elim¬ 
inate  much  of  the  human  involvement  currently  required  in  this 
process.  The  approach  is  illustrated  by  application  of  the 
paradigm  to  the  game  of  hearts.  Recommendations  suggest 
increased  emphasis  on  core  research  problems  standing  between 
current  technology  and  the  capability  of  automatic  knowledge 
programming  and  refinement. 


125.  Klahr,  P.  Conditional  answers  in  question-answering  systems. 

Proceedings  of  the  Sixth  International  Joint  Conference  on 
Artificial  Intelligence,  Tokyo,  Japan,  1979,  461-483. 

In  many  cases  a  deductive  question-answering  system  cannot  find 
complete  proofs  to  answer  questions  requiring  deductive  support. 
In  some  cases  information  needed  to  complete  proofs  is  missing 
from  the  knowledge  base.  In  other  cases  processing  limits  may 
have  been  reached  before  proofs  could  be  completed.  Rather  than 
disregarding  such  partial  proofs  as  most  systems  do,  the  DADM 
system  displays  them  to  users  and  identifies  subproblems  that 
remain  unresolved.  Answers  emanating  from  partial  proofs 
include  remaining  subproblems  as  "conditions"  which  must  be  true 
for  the  answers  to  be  valid. 
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126.  Klahr,  P. ,  and  Faught,  W.  S.  Knowledge-based  simulation.  Proceed¬ 
ings  of  the  First  Annual  National  Conference  on  Artificial 
Intelligence.  Palo  Alto,  1980,  181-183. 

Knowledge  engineering  has  been  successfully  applied  in  many 
domains  to  create  knowledge-based  "expert"  systems.  We  have 
applied  this  technology  to  the  area  of  large-scale  simulation 
and  have  implemented  ROSS,  a  Rule-Oriented  Simulation  System, 
that  simulates  military  air  battles.  Alternative  decision¬ 
making  behaviors  have  been  extracted  from  experts  and  encoded  as 
object-oriented  rules.  Browsing  of  the  knowledge  and  explana¬ 
tion  of  events  occur  at  various  levels  of  abstraction. 


127.  Klahr,  P.,  Faught,  W.  S.,  and  Martins,  G.  R.  Rule-oriented  simula¬ 
tion.  Proceedings  of  the  International  Conference  on  Cybernet  - 
ics  and  Society,  Boston,  1980,  350-354. 

Large-scale  simulators  have  been  plagued  with  problems  of  intel¬ 
ligibility,  modifiability,  credibility  and  performance.  Tech¬ 
niques  developed  in  the  area  of  Artificial  Intelligence,  partic¬ 
ularly  in  the  development  of  knowledge-based  "expert"  systems, 
ate  relevant  to  the  design  of  simulators  that  model  decision¬ 
making  behaviors.  We  outline  the  problems  we  have  encountered 
in  large-scale  simulation  and  present  an  approach  that  incor¬ 
porates  and  extends  those  techniques.  We  describe  ROSS,  a 
Rule-Oriented  Simulation  System,  that  embodies  these  techniques 
in  the  domain  of  military  air  battles. 


128.  Klahr,  P.,  Travis,  L. ,  and  Kellogg,  C.  A  deductive  system  for 

natural -language  question  answering.  In  L.  Bole  (ed.),  Natural 
language  question  answering  systems ,  Hanser-Verlag,  Munich, 

1980,  73-136. 

This  paper  describes  a  deductive  processor  designed  to  enhance 
the  capabilities  of  the  CONVERSE  natural -language  question¬ 
answering  system.  The  emphasis  is  on  deduction  within  a 
question-answering  context  rather  than  within  a  mathematical 
inference  system.  The  deductive  processor  has  been  designed  to 
find,  for  a  given  input  question,  the  relevant  general  premises 
needed  for  deducing  an  answer.  Selecting  relevant  premises  is 
most  crucial  when  there  exists  a  very  large  number  of  premises, 
most  of  which  are  irrelevant  to  any  particular  query.  The 
deductive  system  first  constructs  preliminary,  skeletal  deriva¬ 
tion  proposals.  The  purpose  of  these  proposals  is  to  find  pos¬ 
sible  deductions  before  any  attempt  is  made  to  verify  the  propo¬ 
sals.  Verification  is  thus  delayed  until  global  proof  plans 
have  been  established.  Later  processing  phases  examine  the 
variable  flows  (substitutions)  within  a  proposal  to  detect  pos¬ 
sible  collisions  and  search  the  fact  file  for  compatible  sets  of 
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values  for  instantiations.  The  use  of  semantic  information  is 
incorporated  into  the  system. 


»  • 


129.  Lenat,  D.  B.,  Hayes-Roth,  F. ,  and  Klahr,  P.  Cognitive  Economy. 

Rand  Note  N-1185-NSF,  June  1979. 

Intelligent  systems  can  explore  only  tiny  subsets  of  their 
potential  external  and  conceptual  worlds.  To  increase  their 
effective  capacities,  they  must  develop  efficient  forms  of 
representation,  access,  and  operation.  In  this  Note  we  develop 
several  techniques  which  do  not  sacrifice  expressibility ,  yet 
enable  programs  to  (semi-)autoraatical ly  improve  themselves  and 
thus  increase  their  productivity.  The  basic  source  of  power  is 
the  ability  to  predict  the  way  that  the  program  will  be  used  in 
the  future,  and  to  tailor  the  program  to  expedite  such  uses. 
Abstraction,  caching,  and  expectation-simplified  processing  are 
principal  examples  of  such  techniques.  We  discuss  the  use  of 
these  and  other  economic  principles  for  modern  AI  systems.  Our 
analysis  leads  to  some  counterintuitive  ideas  and  proposed  poli¬ 
cies  which  are  not  generally  followed  because  their  contribution 
to  overall  cognitive  utility  is  not  readily  apparent.  For  exam¬ 
ple,  we  challenge  the  typical  practice  of  storing  properties 
"economically"  in  hierarchical  inheritance  nets.  That  nonredun- 
dant  storage  practice  provides  some  storage  cost  savings,  but 
significantly  increases  processing  costs.  As  an  alternative,  we 
suggest  storage  schemes  to  improve  performance  by  exploiting 
various  forms  of  redundancy  consistent  with  general  caching 
heuristics . 


130.  McArthur,  D.  Intelligent  problem  solving  in  chess .  Ph.D  Thesis, 
University  of  Michigan,  November  1980. 


131.  McArthur,  D.  Machine  vision  and  human  perception.  Human  Perfor 
mance  Technical  Report  //82,  University  of  Michigan,  January 
1980. 


132.  McArthur,  D.  Causal  reasoning  in  chess.  Cognitive  Science  Series 
working  Paper,  University  of  Michigan,  May  1980. 


t 


133.  Shavelson,  R.  J.,  and  Stasz,  C.  Some  methods  for  representing  the 
structure  of  concepts  in  prose  material.  In  J.  Hartley  (ed.). 
The  psychology  of  written  communication,  selected  readings . 
Nichols,  New  York,  1980. 
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134.  Smith,  H.  T. ,  Waterman,  D.  A.,  and  Faught,  W.  S.  An  office  commun 
ication  system  design.  Rand  Paper  P-6393,  September  1979. 


135.  Stasz,  C.  Ability  and  strategy  differences  in  map  learning.  In  M. 
P.  Friedman,  J.  P.  Das  and  N.  O'Connor  (eds.).  Intelligence  and 
learning.  Plenum  Press,  New  York,  in  press.  Also  issued  as  Rand 
Note  N-1569-0NR,  August  1980. 

This  paper  describes  the  influence  of  individual  differences  in 
abilities  and  subject-selected  techniques  for  learning  maps. 
Verbal  protocols  were  obtained  from  25  subjects  who  differed  in 
psychometrically  measured  spatial  restructuring  and  visual 
memory  abilities.  These  protocols  suggested  a  number  of  learn¬ 
ing  procedures  and  strategies  that  subjects  used  to  focus  atten¬ 
tion,  encode  information  and  evaluate  their  learning  progress 
while  studying  a  map.  High  ability  subjects  differed  from  low 
ability  subjects  in  the  overall  strategies  they  adopted  to 
approach  the  learning  problem,  in  their  use  of  imagery  for 
encoding  spatial  information,  and  in  their  subsequent  recall  of 
spatial  attributes  of  the  map. 


136.  Stasz,  C. ,  and  Thorndyke,  P.  W.  The  influence  of  visual  spatial 
ability  and  study  procedures  of  map  learning  skill.  Rand  Note 
N-1501-ONR,  August  1980. 

This  study  investigated  two  sources  of  individual  differences  in 
map  learning;  abilities  and  learning  procedures.  Twenty-five 
subjects  provided  verbal  protocols  while  learning  two  maps. 
Visual-spatial  ability  was  highly  correlated  with  recall  of  spa¬ 
tial  attributes  of  the  map  and  with  overall  performance,  while 
associative  memory  ability  was  most  correlated  with  verbal 
attribute  recall.  Successful  learners  more  frequently  employed 
procedures  for  encoding  spatial  information  and  assessing  learn¬ 
ing  progress.  However,  high  and  low  ability  subjects  differed 
little  in  the  study  procedures  they  chose.  Although  both  abil¬ 
ity  differences  and  procedure  use  were  important  contributors  to 
performance,  abilities  were  the  strongest  predictors  of  map 
learning.  We  concluded  that:  (1)  using  effective  study  pro¬ 
cedures  influences  map  learning,  and  (2)  high  ability  subjects 
benefit  more  from  using  these  procedures  than  low  ability  sub¬ 
jects. 


137.  Thorndyke,  P.  W.  Distance  estimation  from  cognitive  maps.  Rand 
Report  R-2474-0NR,  November  1979. 

Four  experiments  investigated  map  clutter  as  a  source  of  distor¬ 
tion  in  subjects'  estimates  of  distance.  In  Experiments  1  and 
2,  subjects  estimated  distances  between  pairs  of  points  on  a 
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memorized  map.  In  Experiment  1,  they  learned  relative  distances 
among  cities  incidentally;  in  Experiment  2,  they  learned  these 
distances  intentionally.  In  both  experiments,  estimates 
increased  as  a  linear  function  of  the  number  of  intervening 
points  along  the  judged  path.  In  Experiment  3,  subjects 
estimated  distances  while  viewing  the  map.  With  this  procedure, 
the  effect  of  clutter  was  reduced  but  not  eliminated.  In  Exper¬ 
iment  4,  the  clutter  effect  was  demonstrated  using  subjects' 
pre-exper imental  knowledge  of  U.S.  geography.  Psychophysical 
power  functions  relating  true  to  estimated  distance  provided  a 
good  fit  to  both  memory  and  perception  data.  These  results  sug¬ 
gest  an  analogy  between  perceptual  and  memorial  processes  of 
distance  estimation.  The  estimation  model  providing  the  best 
fit  to  the  data  assumed  that  subjects  perceptually  scan  a  route 
(or  a  mental  image  of  a  route)  from  the  starting  point  to  the 
destination  point  and  use  scan  duration  to  determine  route  dis¬ 
tance  . 


138.  Thorndyke,  P.  W.  Performance  models  for  spatial  and  locational 
cognition.  Rand  Report  R-2676-0SR,  December  1980. 

Summarizes  a  three-year  investigation  of  the  knowledge  and 
processes  people  use  to  lean  and  make  spatial  judgments  in 
large-scale  environments  Experiments  in  map  learning  indicated 
that  both  the  use  effective  study  procedures  and  visual 
memory  ability  determine  success  at  learning  a  map.  All  but 
low-ability  people  benefit  from  training  in  effective  study  pro¬ 
cedures.  Studies  of  people's  procedures  for  accuracy  at 
estimating  distanv.es  on  maps  indicated  that  map  clutter 
increases  subjective  distance  between  two  points.  A  third 
series  of  studies  investigated  differences  in  the  knowledge  peo¬ 
ple  acquire  from  navigation  and  from  map  learning.  Studying  a 
map  leads  to  a  global  representation  of  the  environment,  while 
navigation  provides  a  linear,  or  procedural  representation. 
Navigation  experience  is  optimal  for  estimating  route  distances 
and  orienting  oneself  toward  unseen  locations.  Map  learning  is 
optimal  for  estimating  the  shortest  distance  between  two  points 
and  determining  relative  locations  of  objects. 


139.  Thorndyke,  P.  W.  Spatial  cognition  and  reasoning.  In  J.  Harvey 

(ed.),  Cognition,  social  behavior,  and  the  environment,  Lawrence 
Erlbaum  Associates,  Hillsdale,  New  Jersey,  1980. 


140.  Thorndyke,  P.  W. ,  and  Hayes-Roth,  B.  Differences  in  spatial 

knowledge  acquired  from  maps  and  navigation.  Rand  Note  N-1595- 
ONR,  November  1980. 


This  Note  proposes  models  of  the  spatial  knowledge  people 
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acquire  from  maps  and  navigation  and  the  procedures  required  for 
spatial  judgments  using  this  knowledge.  From  a  map,  people 
acquire  survey  knowledge  encoding  global  spatial  relations. 

This  knowledge  resides  in  memory  in  images  that  can  be  scanned 
and  measured  like  a  physical  map.  From  navigation,  people 
acquire  procedural  knowledge  of  the  routes  connecting  diverse 
locations.  People  combine  mental  simulation  of  travel  through 
the  environment  and  informal  algebra  to  compute  spatial  judg¬ 
ments.  An  experiment  in  which  subjects  learned  an  environment 
from  navigation  or  from  a  map  evaluates  predictions  of  these 
models.  With  moderate  exposure,  map  learning  is  superior  for 
judgments  of  relative  location  and  straight-line  distances  among 
objects.  Learning  from  navigation  is  superior  for  orienting 
oneself  with  respect  to  unseen  objects  and  estimating  route  dis¬ 
tances.  With  extensive  exposure,  the  performance  superiority  of 
maps  over  navigation  vanishes.  These  and  other  results  are  con¬ 
sonant  with  the  proposed  mechanisms . 


141.  Thorndyke,  P.  W.,  and  Hayes-Roth,  B.  Human  processing  of  knowledge 
from  texts;  Acquisition,  integration,  and  reasoning.  Rand 
Report  R-2256-ARPA,  June  1979. 

Decisionmakers  must  frequently  assimilate  a  large  number  of 
facts  from  several  documents,  organize  related  facts  in  memory, 
and  reason  using  the  acquired  knowledge.  Nine  experiments 
investigated  how  people  learn  and  retain  knowledge  in  texts  and 
perform  inferential  reasoning  (using  several  facts  to  generate 
or  verify  conclusions).  Individual  experiments  examined  the 
influence  of  text  structure,  the  learnability  of  individual 
facts,  the  acquisition  of  new  knowledge  conforming  to  a  previ¬ 
ously  learned  structure,  the  integration  of  related  but 
separately  learned  facts  in  memory,  search  and  verification 
processes  for  inferential  reasoning,  and  techniques  for  improv¬ 
ing  the  organization  of  information  in  memory.  Results  are 
presented  in  the  context  of  models  for  knowledge  representation 
and  processing.  A  set  of  principles  for  improving  human  learn¬ 
ing  are  derived,  including  text  formats  that  facilitate 
knowledge  acquisition  and  integration. 


142.  Thorndyke,  P.  W. ,  and  Stasz,  C.  Individual  differences  in  pro¬ 
cedures  for  knowledge  acquisition  from  maps.  Cognitive  Psychol¬ 
ogy,  1980,  Vol.  12,  137-175. 

This  study  investigated  the  procedures  subjects  use  to  acquire 
knowledge  from  maps.  In  Experiment  1,  three  experienced  and 
five  novice  map  users  provided  verbal  protocols  while  attempting 
to  learn  a  map.  The  protocols  suggested  four  categories  of 
processes  that  subjects  invoked  during  learning:  attention, 
encoding,  evaluation,  and  control.  Good  learners  differed  from 


poor  learners  primarily  in  their  techniques  for  and  success  at 
encoding  spatial  information,  their  ability  to  accurately  evalu¬ 
ate  their  learning  progress,  and  their  ability  to  focus  atten¬ 
tion  on  unlearned  information.  An  analysis  of  the  performance 
of  experienced  map  users  suggested  that  learning  depended  on 
particular  procedures  and  not  on  familiarity  with  the  task.  In 
Experiment  2,  subjects  were  instructed  to  use  (a)  six  of  the 
effective  learning  procedures  from  Experiment  1,  (b)  six  pro¬ 
cedures  unrelated  to  learning  success,  or  (c)  their  own  tech¬ 
niques.  The  effective  procedures  set  comprised  three  techniques 
for  learning  spatial  information,  two  techniques  for  using 
self-generated  feedback  to  guide  subsequent  study  behaviors,  and 
a  procedure  for  partitioning  the  map  into  sections.  Subjects 
using  these  procedures  performed  better  than  subjects  in  the 
other  groups.  In  addition,  subjects'  visual  memory  ability 
predicted  the  magnitude  of  the  performance  differential. 

143.  Thorndyke,  P.  W. ,  and  Weiner,  M.  Improving  the  training  and  per¬ 
formance  of  Navy  teams;  Design  for  a  research  program.  Rand 
Report  R-2607,  July  1980. 

A  design  for  a  large-scale  research  program  on  improving  Navy 
team  effectiveness.  The  report  assesses  the  most  critical  Navy 
teams,  identifies  deficiencies  in  the  performance  of  these 
teams,  and  recommends  new  research  that  could  lead  to  signifi¬ 
cant  improvements  in  team  performance.  The  major  conclusions 
are  (1)  teams  performing  time-stressed  decisionmaking  using  sym¬ 
bolic  information  are  most  critical  to  mission  effectiveness  and 
ship  survivability,  (2)  several  currently  available  instruc¬ 
tional  methods  could  immediately  improve  training,  (3)  new 
interdisciplinary  research  should  study  simulated  teams  in 
laboratory  task  environments  that  provide  computer -control led 
task  scenarios,  realistic  environmental  and  enemy  models,  and 
intra-team  communications  networks,  and  (4)  promising  approaches 
to  improving  teams  include  improving  performance  models  of  team 
tasks,  improving  the  tools  and  methods  of  training,  compensating 
for  disruptive  effects  of  turnover  in  team  personnel,  improving 
team  organization,  and  improving  human-machine  systems  to  aid 
task  performance. 


144.  Veit,  Clairice  T.  Analyzing  "ratio"  and  "difference"  judgments:  A 
reply  to  Rule  and  Curtis.  Journal  of  Experimental  Psychology: 
General,  1980,  Vol.  109,  No.  3,  301-303. 

This  Note  responds  to  the  speculations  of  Rule  and  Curtis  that 
two  operations  may  underlie  Veit's  "ratio"  and  "difference" 
data.  Rule  and  Curtis  attempted  to  simulate  Veit's  data  by  the 
theory  that  judges  perform  two  operations  corresponding  to  task 
instructions.  They  were  able  to  transform  both  differences  and 
ratios  by  separate  weak  monotonic  transformations  to  the  same 


order  and  scale.  From  this  they  concluded  that  Veit's  data  may 
have  been  so  generated  and  thus  her  conclusion  that  respondents 
use  only  one  operation  for  "ratios"  and  "differences"  is 
suspect.  However,  reanalysis  shows  that  the  simulated  data  do 
not  approximate  Veit's  data  and  that  Veit's  techniques  would 
have  led  to  a  correct  diagnosis  of  Rule  and  Curtis's  simulated 
data.  Other  theories  postulated  by  Rule  and  Curtis  that  experi¬ 
mental  procedures  may  influence  rank  orders  of  "differences"  and 
"ratios"  so  as  to  cancel  the  expected  nonmonotonic  relationship 
between  the  two  response  matrices  are  also  discussed. 


145.  Waterman,  D.  A.  Rule-based  expert  systems.  Rand  Paper  P-6545, 
September  1980. 


146.  Waterman,  D.  A.  User-oriented  systems  for  capturing  expertise:  A 
rule-based  approach.  In  D.  Michie  (ed.).  Expert  systems  in  the 
micro-electronic  age.  1979. 

User-oriented  systems  for  capturing  expertise  are  discussed  and 
illustrated  within  the  context  of  RITA  and  ROSIE,  two  rule-based 
systems  developed  for  building  intelligent  interfaces  and  model¬ 
ing  expert  knowledge.  RITA,  the  Rule-Directed  Interactive  Tran¬ 
saction  Agent  system  is  operational  and  is  currently  being 
applied  to  interface  and  modeling  problems.  ROSIE,  the  Rule- 
Oriented  System  for  Implementing  Expertise,  is  now  under 
development,  and  is  intended  as  a  tool  for  model  builders  work¬ 
ing  in  complex  domains  for  which  useful  analytic  models  are  una¬ 
vailable.  This  note  is  designed  to  promote  discussion  with  col¬ 
leagues  interested  in  rule-based  heuristic  modeling  and  is 
intended  for  a  technical  audience. 


147.  Waterman,  D.  A.,  Anderson,  R.,  Hayes-Roth,  F.,  Klahr,  P. ,  Martins, 
G. ,  and  Rosenschein,  S.  Design  of  a  rule-oriented  system  for 
implementing  expertise.  Rand  Note  N-1158-1-ARPA,  May  1979. 

This  Note  describes  the  preliminary  design  of  a  Rule-Oriented 
System  for  Implementing  Expertise  (ROSIE) .  This  system  is 
intended  as  a  tool  for  model  builders  seeking  to  apply  expert 
knowledge  to  the  analysis  of  problems  and  to  the  evaluation  of 
solutions  in  complex  domains,  especially  domains  for  which  use¬ 
ful  analytic  models  are  unavailable. 

This  preliminary  design  is  the  result  of  a  six-month  design 
exercise,  and  formed  the  basis  of  a  proposal  for  implementation 
of  this  software  system  submitted  to  the  Information  Processing 
Techniques  Office  of  the  Defense  Advanced  Research  Projects 
Agency.  This  Note  is  being  distributed  to  promote  discussion 
and  exchange  of  views  with  colleagues  interested  in  rule- 
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directed  systems  for  heuristic  modeling.  It  is  intended  for  a 
technical  audience;  basic  knowledge  of  the  architecture  of 
rule-based  systems  is  assumed. 


148.  Waterman,  D.  A.,  Faught,  W.  S.,  Klahr,  P.,  Rosenschein,  S.  J. ,  and 
Wesson,  R.  Design  issues  for  exemplary  programming.  Rand  Note 
N-1484-RC,  April  1980. 

This  report  describes  the  design  and  implementation  of  the  Exem¬ 
plary  Programming  (EP)  system  that  allows  software  to  be  created 
by  example.  The  EP  paradigm  is  as  follows:  The  user  performs 
some  interactive  task  on  a  computer.  The  EP  system  watches  over 
the  user's  shoulder,  recording  the  interaction  between  the  user 
and  the  system  he  is  using.  When  the  task  is  done,  EP  con¬ 
structs  an  algorithm  or  high-level  model  of  the  interaction. 

Part  of  this  construction  may  involve  questions  to  the  user  or 
advice  from  the  user.  EP  then  constructs  a  program  (agent)  from 
the  model  and  stores  it  in  a  library  for  subsequent  use.  A  cri¬ 
tique  and  suggestions  for  the  next  version  of  the  EP  system  are 
included. 


149.  Waterman,  D.  A.,  and  Peterson,  M.,  Rule-based  models  of  legal 

expertise.  AAAI  Conference  Proceedings ,  August  1980,  272-275. 

This  paper  discusses  the  design,  implementation  and  testing  of  a 
prototype  legal  decisionmaking  system  (IDS).  This  system  is 
being  implemented  in  ROSIE,  a  rule-oriented  language  designed  to 
facilitate  the  development  of  large  expert  systems.  The  effect 
of  changes  in  the  legal  system  on  settlement  decisions  are  stu¬ 
died  by  modifying  the  rules  representing  formal  legal  doctrine 
and  noting  the  effect  on  settlement  outcomes.  Our  current 
implementation  of  IDS  consists  of  approximately  90  rules 
representing  negligence  and  liability  laws.  Given  a  description 
of  a  product  liability  case  the  model  attempts  to  determine  what 
theory  of  liability  applies,  whether  or  not  the  defendant  is 
liable,  how  much  the  case  is  worth,  and  what  an  equitable  value 
for  settlement  would  be.  Our  preliminary  work  with  IDS  has 
demonstrated  the  feasibility  of  applying  rule-based  modeling 
techniques  to  the  product  liability  area.  Both  the  basic  con¬ 
cepts  needed  to  describe  the  domain  and  the  rules  required  to 
represent  legal  doctrine  and  strategies  can  be  adequately 
represented  in  a  rule-based  formalism  such  as  ROSIE. 
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An  investigation  of  potential  organizations  for  automated  dis¬ 
tributed  sensor  networks,  i.e.,  dispersed  nodes  that  can  pool 
their  information  to  perform  accurate  situation  assessment. 
Laboratory  experiments  using  a  message  puzzle  task  indicate  that 
an  "anarchic  committee"  organization,  in  which  all  nodes  commun¬ 
icate  with  one  another,  consistently  outperforms  the  "dynamic 
hierarchical  cone"  organization,  in  which  communication  is  con¬ 
strained  and  information  must  be  obtained  only  from  lower- level 
nodes.  These  experiments  support  the  contention  that  DSNs  must 
emphasize  cooperative  problem-solving  rather  than  problem- 
reduction  or  subgoaling.  A  computer-based  design  that  minimizes 
redundant  communications  in  hierarchical  organizations  by  using 
model-based  reasoning  to  form  expectations  that  guide,  limit, 
and  reduce  reporting  frequency  is  described.  Finally,  a  method 
for  representing  hypotheses  to  minimize  communication 
requirements- -the  process  assembly  network--is  suggested.  This 
concept  uses  active  "hypotheses  processes"  that  are  responsible 
for  predicting  their  own  evolution  over  time. 

151.  Yekovich,  R.,  and  Thorndyke,  P.  An  evaluation  of  alternative 
functional  models  of  narrative  schemata.  Rand  Paper  P-6299, 
1980. 

This  paper  comparatively  evaluates  several  models  of  how  people 
encode  and  retrieve  information  from  short  narrative  texts.  The 
models  are  distinguished  by  their  detailed  assumptions  on  each 
of  four  issues:  (1)  whether  or  not  propositions  are  encoded 
with  differential  probability  as  a  function  of  their  importance 
in  a  narrative  structure,  (2)  whether  the  representation  of  the 
text  in  memory  is  hierarchical  or  heterarchical,  (3)  whether 
memory  retrieval  depends  primarily  on  direct  access  or  on  a 
top-down  search  process,  and  (4)  whether  or  not  lexical  informa¬ 
tion  is  retained  in  memory.  Specific  assumptions  are  combined 
to  form  twenty  alternative  memory  models.  These  models  are 
evaluated  in  an  experiment  in  which  subjects  attempted  to  learn 
four  narrative  texts.  Memory  for  the  texts  was  tested  by  both 
recall  and  recognition  either  immediately  or  after  a  one-hour 
delay.  Propositional  recall,  but  not  recognition,  varied  as  a 
function  of  importance.  Further,  on  the  recognition  test  sub¬ 
jects  were  able  to  distinguish  statements  that  had  occurred  in 
the  texts  from  meaning-preserving  paraphrases  of  these  state¬ 
ments.  These  data  suggest  that  (1)  subjects  retain  lexical 
information  in  memory,  (2)  narrative  schemata  provide  a  frame¬ 
work  for  encoding  all  text  propositions,  (3)  these  schemata  are 
hierarchically  organized,  and  (4)  the  schemata  are  used  for 
top-down  retrieval  of  information  at  output  time. 
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